Skip to content

fix: defer anti-replay window update until after record authentication#79

Merged
algesten merged 4 commits intoalgesten:mainfrom
HMBSbige:fix/anti-replay-window-before-auth
Mar 7, 2026
Merged

fix: defer anti-replay window update until after record authentication#79
algesten merged 4 commits intoalgesten:mainfrom
HMBSbige:fix/anti-replay-window-before-auth

Conversation

@HMBSbige
Copy link
Copy Markdown
Contributor

@HMBSbige HMBSbige commented Mar 6, 2026

Summary

  • Split ReplayWindow::check_and_update() into check() (read-only) and update() (mutating)
  • Move replay_update() call to after successful decryption in both DTLS 1.2 and 1.3 incoming record paths
  • Prevent DoS attack where forged ciphertext with far-ahead sequence number could shift the window and reject legitimate packets

RFC References

  • DTLS 1.2RFC 6347 §4.1.2.6: "The receive window is updated only if the MAC verification succeeds."
  • DTLS 1.3RFC 9147 §4.5.1: "The window MUST NOT be updated due to a received record until that record has been deprotected successfully."

Test plan

  • New unit tests check_does_not_modify_window and failed_auth_does_not_advance_window verify the two-phase behavior
  • All existing tests pass (cargo test — 69 tests + 8 doc-tests)

Closes #78

Note

Based on #77 (dtls12 branch) to avoid merge conflicts.

HMBSbige and others added 3 commits March 7, 2026 14:49
Split ReplayWindow::check_and_update() into check() (read-only) and
update() (mutating), and call update() only after successful decryption.

Previously the replay window was advanced before verifying the record's
authenticity, allowing an attacker to forge a ciphertext with a far-ahead
sequence number, shift the window, and cause legitimate packets to be
rejected as replays (DoS).

DTLS 1.2 — RFC 6347 §4.1.2.6: "The receive window is updated only if
the MAC verification succeeds."
DTLS 1.3 — RFC 9147 §4.5.1: "The window MUST NOT be updated due to a
received record until that record has been deprotected successfully."

Closes algesten#78
@algesten algesten force-pushed the fix/anti-replay-window-before-auth branch from 68da2c1 to 1853195 Compare March 7, 2026 13:51
@algesten algesten merged commit 1193391 into algesten:main Mar 7, 2026
45 checks passed
@algesten
Copy link
Copy Markdown
Owner

algesten commented Mar 7, 2026

Thank you!!!

@HMBSbige HMBSbige deleted the fix/anti-replay-window-before-auth branch March 8, 2026 04:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anti-replay window updated before record authentication (DTLS 1.2 & 1.3)

2 participants